Computer Study of Quantizer Output Spectra 

By G. H. ROBERTSON 

(Manuscript received January 25, 1969) 

This article describes a method for accurately calculating the output 
spectrum oj a quantizer. The method ivas developed for known expressions 
defining the output spectrum oj an arbitrary quantizer with gaussian input 
of arbitrary bandshape. Results obtained for a variety of conditions, 
however, suggest that the calculations are valid even though the input has 
only a minor gaussian component. When sampling is also used, at the 
Nyquist rate or a little higher, the quantizing noise folded into the input 
band is almost flat even when the input bandshape is sharply peaked. When 
interference at the input is increased, the quantizer {preceded by AGC) 
appears to operate like an increasingly noisy linear transducer up to a 
breaking point beyond which its performance (for small signals) degrades 
rapidly and becomes difficult to analyze. 

I. INTRODUCTION 

Several authors have described formulas for calculating the output 
noise spectrum from a quantizer when the input is a gaussian wave- 
form. References 1 through 4 are characteristic and contain representa- 
tive bibliographies. Evaluation of the resulting expressions is difficult 
because they contain multiple infinite sums of terms containing 
Hermite polynomials whose order increases without limit. Conse- 
quently simplifying assumptions are made about the input spectrum 
and quantizer characteristics, or only a few terms are evaluated and 
the rest assumed negligible, to get results. 

This article describes a more fruitful approach in which the Hermite 
polynomials are evaluated in conjunction with other parts of the ex- 
pression such that the combination tends to zero as the order increases 
to infinity. The convergence is slow and many terms are needed to get 
sufficient accuracy in the noise spectrum. It is possible to get results 
even when the quantizer is not linear or symmetrical, and for arbitrary 
input spectrum shapes. 
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For quantizing steps no greater than o- (the rms gaussian compo- 
nent) an interesting and useful result is that even when the input 
spectrum is sharply peaked, if the quantized waveform is also sam- 
pled uniformly at up to a few times the Nyquist rate for the input 
band, the resulting quantizing noise appearing within the input band 
is nearly flat. Many systems can therefore be evaluated quite ac- 
curately with much simpler calculations than those needed to define 
the quantizing noise spectrum. 

Study of quantizers having uniform steps less than o- in amplitude 
show the output spectrum to be practically independent of the loca- 
tion of the gaussian mean if it is at least a from the overload limit. 
Consequently, added signals (whose waveform defines the gaussian 
mean) have a negligible effect on the quantizer output noise as long 
as they do not approach within a of the limit. A quantizer with many 
steps activated thus produces a noise spectrum virtually independent 
of relatively large signals added to the gaussian component. 

II. DEFINITION OF QUANTIZER 

Figure 1 shows the transfer characteristic of the quantizer where 
the "staircase" relates the output voltage (ordinate) scale to the input 
voltage (abscissa) scale. Assuming that the input waveform is gaus- 
sian about some arbitrary mean value, the probability that it is Z or 
more above the mean value is 

Q * = (dj^f exp( - i72 * V( - (1) 

Figure 1 shows that when the input waveform reaches a "riser" of 
the staircase, the output waveform changes abruptly from the value 
on one tread to the value of the one on the other side of the riser. For 
convenience, number the treads and risers starting with 1 at the left. 
There is one more tread than the number of risers, so if the last riser 
is fc, the last tread is k + 1. Let Q r be the probability that the input 
waveform is greater than riser r, and the output voltage of step r be 
W r . The mean value of the output is 

S = Wy{l - QO + W 2 (Q, - Q 2 ) + • • • + W k+1 Q k . (2) 

The mean squared value is 

V 2 = W\(l - Q t ) + WUQx - 00 + • • ■ + Wl+iQ* . (3) 

The variance of the output is 
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V 2 - S 2 = P. 
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(4) 



Assuming unity impedance, P is the output power after subtracting the 
component caused by the displacement of the mean value of the input 
waveform from zero. 



III. QUANTIZING NOISE SPECTRUM 



Velichkin showed that the correlation function of the quantizer out- 
put can be written 4 



B,(r) = E 



(5) 



R x (t) is the input correlation function, a 2 is the input variance, there 
are v treads, A& is the output voltage difference between treads k + 1 
and k, a k is the input voltage at riser k, and H r (z) is the Hermite 
polynomial 



HM = (-l) r exp (z 2 /2) £ r [exp (-V/2)]. 
Also, where [r/2] is the greatest integer ^ r/2, s 



(6) 
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Fig. 1 — Quantizer transfer characteristics. 
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H,(*) = "f (-vs&v-'ji^r (7) 

By the Wiener-Khinchine theorem the power spectrum of the 
quantizer output is 



B0) = 4 f R„(t) cos (2t/t) dr 

•>0 



p » ' I "n-ll 



•'n 



= ; S IS At exp ( ~ al/2a2) [(n^wJ ^ 

fl^r) COS (2t/t) dr. (8) 

'o 

Equation (8) can be written 

00) = E T= 4 r fifr) COS (27r/r) dr, (9) 

n = l O" Jo 

in which the quantizing factor terms F n depend only on the properties 
of the quantizer and n. When n = 1 the component Q (/) is the input 
spectrum multiplied by i*V<r 2 . All other n give components whose 
bandwidth exceeds that of the input [because the integral in equation 
(9) then represents multiple convolutions of the input band], and 
their sum fi e (/) may be called the quantizer error spectrum. The 
quantizer output spectrum is 

00) = 0.(0 + 0.0). (10) 

So far only amplitude quantizing has been considered. Sampling, 
at a rate f g , is generally also used,* and the output spectrum be- 
comes proportional to 1 

0.0) - m + E 0(n/. ± /)• (11) 

n-l 

If /„ is at least twice the highest frequency of the input band, only fi t . 
can add more noise by fold-over into the range of the input band. 

These results are all known but now follow what are thought to be 
new contributions enabling equation (9) to be evaluated for an 
arbitrary choice of input spectrum shape. Equation (9) can be writ- 
ten 



* The result is independent of which is done first. 
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00) = E^2 [* We-""* dr 

= E%2C n _ 1 [ S (/)/2], (12) 

where C„_i[s(/)/2] is the (n— l)th convolution of the cisoid power 
spectrum s(/)/2. The sinusoid power spectrum is s(/), corresponding 
to the autocorrelation function R x {t). 

The significance of equation (12) is that whereas the direct calculation 
of R*(t) may be impossible for an arbitrary spectrum shape, C n _i[s(/)/2] 
can always be calculated if s(/) is defined. Appendix A describes the 
methods used to calculate C„_,[s(/)/2] in the computer program written 
to evaluate equation (9). 

If G{t) represents the input waveform, the autocorrelation func- 
tion at zero lag is 



RM = lim^= I' G(tf 



dt 



= c 2 + S 2 , (13) 

where S is the mean value of the input waveform and o- 2 is the variance 
as used in equation (1). Figure 1 shows that the value of the quantizer 
output for a given input waveform is independent of the scale on the 
input axis. For convenience, relabel this scale so that the input mean 
is zero. Consequently, 

RM = * 2 . (14) 

Normalizing the input power that now contains no dc, so that a 2 = 
1, gives 

R'M = 1 (15) 

for all n. By the Wiener-Khinchine theorem the total output power 
P r is 

P T = r W dj 

-'o 

= RM 

= hU{i^^^ /2) i^%l (i6) 

using equation (5). Pt is the same as T /2 given by equation (3) so the 
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accuracy of computing the quantizing factor terms F n can be checked 
by computing the total power by both these methods. The method of 
equation (3) gives high accuracy very easily, but the F n terms are 
needed to compute the quantizer error spectrum ft e (/) of equation 
(10). Appendix B describes the methods used to compute F n for values 
of n up to 10,000, the limit used in the program. 

Recall [after equation (9)] that when n = 1 the resulting component 
of 0(/) is the input spectrum multiplied by Fi/a 2 , where the gain 
factor is 

Fi = £ [g A. exp (-a*/2<r 2 )] 2 . (17) 

When a* = 1, the total quantizer error power is 

P E = P - F lf (18) 

where P is given in equation (4). Both P and F x can be computed 
easily and accurately, so Pe can be determined accurately with little 
computational effort. Note that this shows Pe to be independent of 
the input spectrum shape. 

A computer program, using the techniques described in Appendixes 
A and B to compute Q, (/) , simulated the effect of sampling (without 
holding) by pivoting Sl(/) about the sampling frequency and its 
harmonics, and computing the contributions thus folded into the 
original band. The total P E is folded into a bandwidth equal to half 
the sampling frequency; and when the latter was less than a few 
times the Nyquist rate for the input band, the level of the error com- 
ponent resulting from P B was nearly flat over the input band even 
when the input spectrum was sharply peaked. 

This result is very useful because the performance of quantizers can 
now be evaluated quite accurately using only the simple calculations 
indicated by equations (4) and (17). The error spectrum after sam- 
pling was flatter when more levels were used in the quantizer. 

IV. SIGNALS ADDED TO INPUT 

Signals added to the gaussian noise at the input cause the mean 
value of the latter to vary according to the signal waveform. Com- 
putation shows that under static conditions the gain factor F^ and 
the total error power P E remain nearly constant when the step size 
is about <r and the mean is no closer than <r to the overload limit. 
Under these conditions the position of the mean has negligible effect 
on the shape of the quantizing noise spectrum. Assuming a signal wave- 
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form uncorrelated with the gaussian noise, and of a magnitude such 
that the mean rarely approaches within a- of the overload limit, it is 
thus quite accurate to assume that the quantizing error is independent 
of the signal when the step-to-o- ratio is constant and less than unity. 
A sampling rate, up to a few times the high end of the input band, 
further improves the accuracy of this assumption as the quantizing 
noise then becomes almost flat across the input band even when the 
input spectrum is sharply peaked. 

Assume now that an AGC unit is used to maintain constant power 
into the quantizer so that the waveform representing the sum of the 
gaussian component and large signal (interference) very rarely ex- 
ceeds the overload limits. As the level of the interference increases, 
the ratio of quantizer step to gaussian rms (rms 5 ) also increases. 
Assuming no correlation between the interference and gaussian com- 
ponents, the degradation from quantizing noise can be estimated from 
the way the parameters i^i and P E vary with the position of the mean. 
The greatest variation in these parameters occurs between the values 
when the mean is at a riser (see Fig. 1) and when it is midway be- 
tween risers. 

Figures 2 and 3 show the results obtained for a 16-level quantizer 
with a flat input spectrum and with a sharply peaked input spectrum, 
respectively, in calculations carried out for these limiting cases. Up to 
a breaking point (where the two curves diverge) the quantizer ap- 
pears to act like a linear but noisy transducer for input signals. Note 
that the breaking point seems to be independent of the spectrum 
shape. When the interference level is high enough to cause operation 
beyond the breaking point, the spectrum becomes difficult to analyze 
and depends on the interference waveform. At all points on the abscis- 
sas of Figs. 2 and 3 below the breaking point, F x and P E were found 
virtually constant for all positions likely to be occupied by the input 
mean (determined by the AGC unit) . Since the quantizing noise level 
was flat it was therefore proportional to P E - The input copy was 
proportional to Fx ; the curves in Figs. 2 and 3 show the ratio of the 
level of input copy plus quantizing noise to the level of the input copy 
alone. The degradation these curves indicate, as the interference in- 
creases, results from the decreasing ratio of a to quantizing step size 
caused by the AGC unit preceding the quantizer. 

V. COMPARISON WITH MEASUREMENTS 

A sharply peaked spectrum was produced in the laboratory by 
filtering the output of a noise generator, and the resulting waveform 
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Fig. 2 — Quantizer performance (static) versus relative interference (flat input) 
for 16-level quantizer with flat input spectrum (gaussian) ; overload at 3 X (MS g 
+ MSi)K set by AGC; MS g = gaussian component (rms) 2 ; MSi = interference 
(rms) 2 ; output sampled at 3 X high end. 

was radically clipped before being submitted to a spectrum analyzer. 
A 1910-A recording wave analyzer (made by General Radio Company) 
was used, and several successive traces were superimposed by the 
recorder as the narrowband (10 Hz) filter was slowly swept across 
the spectrum. The spectrum before and after clipping were determined 
in this way ; the final results were obtained by drawing a smooth curve 
through the mean of the superimposed traces. Figure 4, where the solid 
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Fig. 3 — Quantizer performance (static) versus relative interference (peaked 
input) for 16-level quantizer with peaked input spectrum (gaussian) ; overload 
at 3 X (MS, + MS,) % set by AGC. 
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curve is the computed clipper output spectrum when a copy of the 
input is given by the dashed curve, shows the results. Values of the 
measured output spectrum appear as circles and agree well with the 
computed curve. 

Another check between computed and measured results can be 
obtained for a uniform step 16-level quantizer. A band of noise, nearly 
flat from zero to about 330 Hz and falling rapidly at higher frequencies, 
is added to a sinewave at 160 Hz and passes through an AGC unit 
before quantization. The quantizer overload limit is set near four times 
the rms value of the AGC output, and the results are recorded on a 
magnetic tape for various ratios of the sinewave-to-noise power. In 
this capacity the sine wave acts as an interfering signal. A computer 
program processes the tape using a version of the east Fourier trans- 
form algorithm to produce estimates of the spectrum level at the 
quantizer output up to half the sampling rate of 1024 Hz. 8 Since the 
input spectrum level at 500 Hz is much lower than in the flat part- 
below 300 Hz, the increase in noise level estimated at 500 Hz is taken 
as a measure of the quantizing noise introduced as the interfering 
signal increases. Assuming this noise to be flat from 512 Hz to zero it 
is possible to estimate the degradation in signal-to-noise power suf- 
fered by a small signal in the flat part of the input band. 

Figure 5 shows the results, as circles superimposed on the solid 
curves, which are computed for a 16-level quantizer sampled at three 
times the high end of an input band of noise flat to zero frequency. 
The quantizer is preceded by an AGC unit and its overload is four 
times the rms input. 
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Fig. 4 — Clipper output spectrum. 
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Fig. 5 — Quantizer degradation for flat input spectrum for 16-level quantizer ; 
overload = 4 X input (rms) set by AGC; flat input gaussian component (0 to 
330 Hz) ; sinewave interference (160 Hz) ; output sampled at 1024 Hz. 

VI. CONCLUSION 

This article describes a new method of calculating the quantizing 
noise spectrum when gaussian noise with arbitrary spectrum shape is 
applied to an arbitrary quantizer. The novelty is not in the form of the 
expressions that describe the noise spectrum but in the techniques used 
to compute the results. Applying the method to a sharply peaked spec- 
trum shows that if the output is sampled at the Nyquist rate, or a little 
higher, the quantizing noise is folded back to cover the input band 
with almost uniform intensity. A clipper (2-level quantizer) and a 
16-level quantizer, preceded by AGC to keep the overload at three 
times the rms input, operate like noisy but linear transducers for 
added signals of power less than one tenth and less than twenty times, 
respectively, that of the broadband background. These useful results 
indicate that the performance of quantizers under such conditions 
can be evaluated without the lengthy computations required to deline- 
ate the quantizing noise spectrum. 



APPENDIX A 



Calculating the Input Spectrum Convolutions 

The input cisoid spectrum is defined and convolved with itself to 
calculate C„_i[s(/)/2] when n is small. Because the input spectrum 
is of finite width, the convolutions tend to take the form of a gaussian 



QUANTIZER OUTPUT SPECTRA 2401 

distribution as n increases. Since direct computation of the convolu- 
tions becomes very lengthy when n is large, it is profitable to compute 
a Gram-Charlier approximation instead (see pp. 257-260 of Ref. 5.) 
This can be done if the moments for the desired convolution can be 
obtained. The input cisoid spectrum is symmetrical about zero, and is 
defined up to its limiting bandwidth, so all the moments desired can 
be computed for it. If the input spectrum shape is normalized so that 
it covers unit area and it is considered to define a probability dis- 
tribution from which random samples are drawn, the nth convolution 
is the same as the probability distribution of (n + 1) independent 
samples of the original distribution. 7 The moments of the nth convolu- 
tion can thus be obtained from the moments of the input spectrum 
shape as follows. Since we desire ultimately standardized central 
moments, note that the standardized central moments for the sum 
and for the average of N independent samples are the same. Using the 
appropriate multinomial expansion the general term for the vth such 
moment is 

where 

v = ip + jq + kr, (20) 

the right side being a partition of v, and* 

/ = i + j -f k - 1. 

The sum is taken over all the partitions of v except those containing 
unity (because the first central moment is zero). The term ft p is the 
pth standardized central moment of the original distribution. A pro- 
gram was developed to compute such moments; but since the com- 
putation rapidly becomes very lengthy when v increases, the number 
of moments used to get the Gram-Charlier approximation was re- 
duced as the convolution order increased. This can be done without 
undue sacrifice in accuracy since the distribution tends to become 
gaussian with increasing convolution order. 

APPENDIX B 

Calculation of Quantizer Factor Terms F n 

Equation (8) shows that F„ requires computation of terms like 

* A partition of v is a set of positive integers whose sum is v. The terms i, j, k, 
p, q, and r are integers. 
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F kn = exp {-xl/2)H n .,{x k )/[{n - 1) !]*, 



(21) 



where H r (x) is a Hermite polynomial for which the recurrence rela- 
tion exists 8 



H r+l (x) = xH r (x) - rH r ^(x). 
Therefore 

F«. +1 , = x k FJn* - n u _„[(n - l)/n]*. 
Since H {x) = 1 and Hi{x) = x, from equation (21) 

F u = exp (-xl/2) 

and 

F k2 = x k F kl . 



(22) 
(23) 

(24) 
(25) 



Therefore, by using equations (23), (24), and (25), a straightfor- 
ward method exists for finding any F kn . When values are to be calcu- 
lated using the same x k and many successive values of n, the pro- 
gramming can be simplified by saving the computed values for n and 
{n — 1) to be used in equation (23) when the value for (n + 1) is 
desired. Taking advantage of this way of arranging the computa- 
tions values were computed for n up to 10,000, enabling determina- 
tion of the quantizing noise level at greater than 100 times the input 
bandwidth for a 16-level quantizer. Since recurrence relations like 
that in equation (23) sometimes result in rapid loss of accuracy, a 
few values of F kn were computed by an independent method, for high 
values of n. 

Hermite polynomials can be evaluated in terms of confluent hy- 
pergeometric functions; 8 a suitable asymptotic formula for these 







Table I — Values of 


F kn 


Xk 


n 


Recurrence Relation 


Asymptotic Formula 


1.0 
1.0 
1.0 


9999 
10,000 
10,001 


0.3540125940E-01 
0.60060871554397E-01 
-0.34798910623E-01 


0.35401259262E-01 
. 60060871554399E-01 
-0.34798910644E-01 


2.0 
2.0 
2.0 


9999 
10,000 
10,001 


0.28830572153E-01 
0.16057291188981E-01 
-0.28508000965E-01 


0.28830572171E-01 
0. 16057291 188984E-01 
-0.28508000983E-01 


10.0 
10.0 
10.0 


9,999 
10,000 
10,001 


-0.62935376617E-12 
0.1037121050651E-12 
0.73302922069E-12 


-0.629353766E-12 
0.1037121050655E-12 
0. 733029221 15E-12 



QUANTIZER OUTPUT SPECTRA 2403 

functions was obtained in Ref. 9. Although the asymptotic formula 
would give adequate accuracy when n is large, the recurrence relation 
permits much faster evaluations when values are needed over a large 
range of n. Table I compares a few values of F kn calculated by the 
recurrence relation and the asymptotic formula. Very good agreement 
is obtained justifying the use of the recurrence relation. 
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